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Abstract 

Motivated by social balance theory, we develop a theory of link classification in signed net- 
works using the correlation clustering index as measure of label regularity. We derive learning 
bounds in terms of correlation clustering within three fundamental transductive learning set- 
tings: online, batch and active. Our main algorithmic contribution is in the active setting, 
where we introduce a new family of efficient link classifiers based on covering the input graph 
with small circuits. These are the first active algorithms for link classification with mistake 
bounds that hold for arbitrary signed networks. 

1 Introduction 

Predictive analysis of networked data — such as the Web, online social networks, or biological 
networks — is a vast and rapidly growing research area whose applications include spam detec- 
tion, product recommendation, link analysis, and gene function prediction. Networked data are 
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typically viewed as graphs, where the presence of an edge reflects a form of semantic similarity 
between the data associated with the incident nodes. Recently, a number of papers have started 
investigating networks where links may also represent a negative relationship. For instance, disap- 
proval or distrust in social networks, negative endorsements on the Web, or inhibitory interactions 
in biological networks. Concrete examples from the domain of social networks and e-commerce 
are Slashdot, where users can tag other users as friends or foes, Epinions, where users can give pos- 
itive or negative ratings not only to products, but also to other users, and Ebay, where users develop 
trust and distrust towards agents operating in the network. Another example is the social network 
of Wikipedia administrators, where votes cast by an admin in favor or against the promotion of 
another admin can be viewed as positive or negative links. The emergence of signed networks has 
attracted attention towards the problem of edge sign prediction or link classification. This is the 
task of determining whether a given relationship between two nodes is positive or negative. In 
social networks, link classification may serve the purpose of inferring the sentiment between two 
individuals, an information which can be used, for instance, by recommender systems. 

Early studies of signed networks date back to the Fifties. For example, [jl4| and |j3l model 
dislike and distrust relationships among individuals as negatively weighted edges in a graph. The 
conceptual context is provided by the theory of social balance, formulated as a way to understand 
the origin and the structure of conflicts in a network of individuals whose mutual relationships can 
be classified as friendship or hostility [15|. The advent of online social networks has witnessed 
a renewed interest in such theories, and has recently spurred a significant amount of work — see, 
e.g., [[TSldllllOllTlIIil, and references therein. According to social balance theory, the regularity 
of the network depends on the presence of "contradictory" cycles. The number of such bad cycles 
is tightly connected to the correlation clustering index of [IJ. This index is defined as the smallest 
number of sign violations that can be obtained by clustering the nodes of a signed graph in all 
possible ways. A sign violation is created when the incident nodes of a negative edge belong to the 
same cluster, or when the incident nodes of a positive edge belong to different clusters. Finding 
the clustering with the least number of violations is known to be NP-hard [I J. 

In this paper, we use the correlation clustering index as a learning bias for the problem of 
link classification in signed networks. As opposed to the experimental nature of many of the 
works that deal with link classification in signed networks, we study the problem from a learning- 
theoretic standpoint. We show that the correlation clustering index characterizes the prediction 
complexity of link classification in three different supervised transductive learning settings. In 
online learning, the optimal mistake bound (to within logarithmic factors) is attained by Weighted 
Majority run over a pool of instances of the Halving algorithm. We also show that this approach 
cannot be implemented efficiently under standard complexity-theoretic assumptions. In the batch 
(i.e., train/test) setting, we use standard uniform convergence results for transductive learning ||9l to 
show that the risk of the empirical risk minimizer is controlled by the correlation clustering index. 
We then observe that known efficient approximations to the optimal clustering can be used to 
obtain polynomial-time (though not practical) link classification algorithms. In view of obtaining 
a practical and accurate learning algorithm, we then focus our attention to the notion of two- 
correlation clustering derived from the original formulation of structural balance due to Cartwright 
and Harary. This kind of social balance, based on the observation that in many social contexts 
"the enemy of my enemy is my friend", is used by known efficient and accurate heuristics for 
link classification, like the least eigenvalue of the signed Laplacian and its variants [1I81I . The two- 
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Figure 1: An edge-labeled undirected graph {G, Y) whose nodes V = {1, . . . , 12} are partitioned 
in three clusters. Black solid edges are positive, red dashed edges are negative. The cost of the 
displayed partition is three due to edges (5, 6), (8, 11), and (10, 12). Hence A(F) < 3. There are 
two bad cycles: 8 — 9 — 10 — 11 and 10 — 11 — 12, hence A(F) cannot be zero. In fact, in order to 
remove all bad cycles, it suffices to delete edge (10, 11). By Proposition [T| this yields A(F) = 1. 
A corresponding optimal partition of V is {{1, 2, 3, 4}, {6}, {5, 7, 8, 9, 11, 12}, {10}}. 

correlation clustering index is still hard to compute, but the task of designing good link classifiers 
sightly simplifies due to the stronger notion of bias. In the active learning protocol, we show 
that the two-correlation clustering index bounds from below the test error of any active learner on 
any signed graph. Then, we introduce the first efficient active learner for link classification with 
performance guarantees (in terms of two-correlation clustering) for any signed graph. Our active 
learner receives a query budget as input parameter, requires time ( | -E | \/\V\ 1^ I ^ I ) to predict the 
edges of any graph G = (V, E), and is relatively easy to implement. 

2 Preliminaries 

We consider undirected graphs G = (V, E) with unknown edge labeling Yij E { — 1,+1} for 
each G E. Edge labels of the graph are collectively represented by the associated signed 
adjacency matrix Y, where Yij = whenever E. The edge-labeled graph G will hence- 
forth be denoted by {G, Y). Given (G, Y), the cost of a partition of V into clusters is the number 
of negatively-labeled within-cluster edges plus the number of positively-labeled between-cluster 
edges. We measure the regularity of an edge labeling F of G through the correlation clustering 
index A(F). This is defined as the minimum over the costs of all partitions of V. Since the cost of 
a given partition of V is an obvious quantification of the consistency of the associated clustering, 
A(F) quantifies the cost of the best way of partitioning the nodes in V. Note that the number of 
clusters is not fixed ahead of time. In the next section, we relate this regularity measure to the 
optimal number of prediction mistakes in edge classification problems. 

A bad cycle in (G, Y) is a simple cycle (i.e., a cycle with no repeated nodes, except the first 
one) containing exactly one negative edge. Because we intuitively expect a positive link between 
two nodes be adjacent to another positive link (e.g., the transitivity of a friendship relationship 
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between two individuals)!^ bad cycles are a clear source of irregularity of the edge labels. The 
following fact relates A(F) to bad cycles — see, e.g., [8|. Figure [T] gives a pictorial illustration. 

Proposition 1. For all {G^Y), A(F) = iff there are no bad cycles. Moreover, A(F) is the 
smallest number of edges that must be removed from G in order to delete all bad cycles. 

Since the removal of an edge can delete more than one bad cycle, A(F) is upper bounded 
by the number of bad cycles in (G, Y). For similar reasons, A(F) is also lower bounded by the 
number of edge-disjoint bad cycles in (G, Y). We now showj^that Y may be very irregular on 
dense graphs, where A(F) may take values as big as 0(|i?|). 

Lemma 1. Given a clique G = (V, E) and any integer < K < — 3)(|K| — 4), there exists 
an edge labeling Y such that A(F) = K. 

The restriction of the correlation clustering index to two clusters only leads to measure the reg- 
ularity of an edge labeling Y through A2(F), i.e., the minimum cost over all tw o- cluster -pdxtitiom 
of V. Clearly, A2(F) > A(F) for all Y. The fact that, at least in social networks, A2(F) tends 
to be small is motivated by the Cartwright-Harary theory of structural balance ("the enemy of my 
enemy is my friend")]^ On signed networks, this corresponds to the following multiplicative rule: 
Yij is equal to the product of signs on the edges of any path connecting itoj. It is easy to verify 
that, if the multiplicative rule holds for all paths, then ^2{Y) = 0. It is well known that A2 is 
related to the signed Laplacian matrix Lg of {G, Y). Similar to the standard graph Laplacian, the 
signed Laplacian is defined as Lg = D — Y , where D = Diag((ii, . . . , is the diagonal matrix 
of node degrees. Specifically, we have 

4A2(F) . x'^LgX 

= mm 5- . (1) 

n 3:e{-i,+i}" ||a;|| 

Moreover, ^2(Y) = is equivalent to \Ls\ = — see, e.g., [17]. Now, computing ^2{Y) is still 
NP-hard [12]. Yet, because ([T]) resembles an eigenvalue/eigenvector computation, [fTSl and other 
authors have looked at relaxations similar to those used in spectral graph clustering [|25l . If Amin 
denotes the smallest eigenvalue of Lg, then ([T]) allows one to write 

. x-'LgX ^ AA.jY) 
Amin = mm ——2- < • (2) 

a;GlR" \\x\\ n 

As in practice one expects A2(F) to be strictly positive, solving the minimization problem in ^ 
amounts to finding an eigenvector v associated with the smallest eigenvalue of Lg. The least eigen- 
value heuristic builds Lg out of the training edges only, computes the associated minimal eigenvec- 
tor V, uses the sign of v's components to define a two-clustering of the nodes, and then follows this 
two-clustering to classify all the remaining edges: Edges connecting nodes with matching signs 
are classified +1, otherwise they are —1. In this sense, this heuristic resembles the transductive 



' Observe that, as far as A(Y) is concerned, this need not be true for negative Hnks — see Proposition [l| This is 
because the defirution of A{Y) does not constrain the number of clusters of the nodes in V. 
^ Due to space Hmitations, all proofs are given in the appendix. 

^ Other approaches consider different types of local structures, like the contradictory triangles of |fT9]| . or the longer 
cycles used in Q. 
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risk minimization procedure described in Subsection 3.2 However, no theoretical guarantees are 
known for such spectral heuristics. 

When A2 is the measure of choice, a bad cycle is any simple cycle containing an odd number 
of negative edges. Properties similar to those stated in Lemma[T]can be proven for this new notion 
of bad cycle. 



3 Mistake bounds and risk analysis 

In this section, we study the prediction complexity of classifying the links of a signed network in 
the online and batch transductive settings. Our bounds are expressed in terms of the correlation 
clustering index A. The A 2 index will be used in Section |4] in the context of active learning. 

3.1 Online transductive learning 

We first show that, disregarding computational aspects, A(y) + |l^| characterizes (up to log factors) 
the optimal number of edge classification mistakes in the online trandsuctive learning protocol. In 
this protocol, the edges of the graph are presented to the learner according to an arbitrary and 
unknown order ei, . . . , e^, where T = \E\. At each time t = 1, . . . , T the learner receives edge 
et and must predict its label Yt. Then Yt is revealed and the learner knows whether a mistake 
occurred. The learner's performance is measured by the total number of prediction mistakes on 
the worst-case order of edges. Similar to standard approaches to node classification in networked 
data [[T6l l4ll5l[6ll24l. we work within a transductive learning setting. This means that the learner 
has preliminary access to the entire graph structure G where labels in Y are absent. We start by 
showing a lower bound that holds for any online edge classifier, and then we prove how the lower 
bound can be strengthened if the graph is dense. 

Theorem 2. For any G = {V, E), any K > 0, and any online edge classifier, there exists an edge 
labeling Y on which the classifier makes at least \V\ — 1 + K mistakes, while A(y) < K. 

Theorem 3. For any clique graph G = {V, E), any K > 0, and any online classifier, there exists 
an edge labeling Y on which the classifier makes at least \V\ + maxji^', K\og2 ^| + ^2(1) 
mistakes, while A(F) < K. 

The above lower bounds are nearly matched by a standard version space algorithm: the Halving 
algorithm — see, e.g., [21]. When applied to link classification, the Halving algorithm with param- 
eter d, denoted by hal^^, predicts the label of edge as follows: Let St be the number of labelings 
Y consistent with the observed edges and such that A(F) = d (the version space), hal^ predicts 
+1 if the majority of these labelings assigns +1 to Cj. The +1 value is also predicted as a default 
value if either St is empty or there is a tie. Otherwise the algorithm predicts —1. Now consider 
the instance of Halving run with parameter d* = A(F) for the true unknown labeling Y. Since 
the size of the version space halves after each mistake, this algorithm makes at most logg \ S* \ mis- 
takes, where S* = Si is the initial version space of the algorithm. If the online classifier runs the 
Weighted Majority algorithm of [|22|| over the set of at most \E\ experts corresponding to instances 
of HALrf for all possible values d of A(F) (recall Lemma [T]), we easily obtain the following. 
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Theorem 4. Consider the Weighted Majority algorithm using HALi, . . . , haL|e| as experts. The 
number of mistakes made by this algorithm when run over an arbitrary permutation of edges of a 
given signed graph (G, Y) is at most of the order of [\V\ + A(F)) log2 

Comparing Theorem |4] to Theorem [2] and Theorem |3] provides our characterization of the pre- 
diction complexity of link classification in the online transductive learning setting. 

Computational complexity. Unfortunately, as stated in the next theorem, the Halving algorithm 
for link classification is only of theoretical relevance, due to its computational hardness. In fact, 
this is hardly surprising, since A(F) itself is NP-hard to compute [IJ. 

Theorem 5. The Halving algorithm cannot be implemented in polytime unless RP = NP. 

3.2 Batch transductive learning 

We now prove that A can also be used to control the number of prediction mistakes in the batch 
transductive setting. In this setting, given a graph G = {V, E) with unknown labeling Y E 
{— and correlation clustering index A = A(F), the learner observes the labels of a 
random subset of m training edges, and must predict the labels of the remaining u test edges, 
where m + u = \E\. 

Let 1, . . . , m + M be an arbitrary indexing of the edges in E. We represent the random set of m 
training edges by the first m elements Zi, . . . , in a random permutation Z = (Zi, . . . , Zm+u) 
of {1, . . . ,m + u}. Let V(y) be the class of all partitions of V and f E V denote a specific 
(but arbitrary) partition of V. Partition / predicts the sign of an edge t E {l,...,m + M} using 
f{t) E { — 1,+1}, where f(t) = 1 if / puts the vertices incident to the t-th edge of G in the 
same cluster, and —1 otherwise. For the given permutation Z, we let Am(/) denote the cost of 
the partition / on the first m training edges of Z with respect to the underlying edge labeling 
Y. In symbols, Am{f) = J2tLi{fiZt) 7^ Yzt}- Similarly, we define A„(/) as the cost of / 

on the last u test edges of Z, A„(/) = Xmtn+i{/(^t) 7^ ^^t} • consider algorithms that, 
given a permutation Z of the edges, find a partition f E V approximately minimizing Am(/). 
For those algorithms, we are interested in bounding the number Au(/) of mistakes made when 
using / to predict the test edges Z^^i, . . . , Zm+u- In particular, as for more standard empirical 
risk minimization schemes, we show a bound on the number of mistakes made when predicting 
the test edges using a partition that approximately minimizes A on the training edges. 

The result that follows is a direct consequence of [l9l, and holds for any partition that approxi- 
mately minimizes the correlation clustering index on the training set. 

Theorem 6. Let {G, Y) be a signed graph with A(Y) = A. Fix 6 E (0, 1), and let f* E V{V) be 
such that Am(/*) < k min/gp Amif) for some k > 1. If the permutation Z is drawn uniformly 
at random, then there exist constants c, c' > such that 

-u^uih <^A + c^J{i + i){\V\ln\V\+ln|) + c'.^^lnf 
holds with probability at least 1 — 6. 
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We can give a more concrete instance of Theorem [6] by using the polynomial-time algorithm 
of which finds a partition /* G P such that Am(/*) < 3 ln(| V| + 1) min^gp Am(/) .Assuming 
for simplicity u = m = ^\E\, the bound of Theorem |6] can be rewritten as 

A„(r ) < I ln(|y| + 1) A + O [^\E\ {\V\\n\V\+\n^)+ In \V\ ^\E\ In |) . 

This shows that, when training and test set sizes are comparable, approximating A on the training 
set to within a factor In |y | yields at most order of A In + ^y\E\ \V\ \n\V\ errors on the test 
set. Note that for moderate values of A the uniform convergence term ^/\E\ \V\ \n\V\ becomes 
dominant in the boundj^ Although in principle any approximation algorithm with a nontrivial 
performance guarantee can be used to bound the risk, we are not aware of algorithms that are 
reasonably easy to implement and, more importantly, scale to large networks of practical interest. 

4 Two-clustering and active learning 

In this section, we exploit the A2 inductive bias to design and analyze algorithms in the active 
learning setting. Active learning algorithms work in two phases: a selection phase, where a query 
set of given size is constructed, and a prediction phase, where the algorithm receives the labels 
of the edges in the query set and predicts the labels of the remaining edges. In the protocol we 
consider here the only labels ever revealed to the algorithm are those in the query set. In particular, 
no labels are revealed during the prediction phase. We evaluate our active learning algorithms just 
by the number of mistakes made in the prediction phase as a function of the query set size. 

Similar to previous sections, we first show that the prediction complexity of active learning is 
lower bounded by the correlation clustering index, where we now use A2 instead of A. In partic- 
ular, any active learning algorithm for link classification that queries at most a constant fraction of 
the edges must err, on any signed graph, on at least order of A2(F) test edges, for some labeling 
Y. 

Theorem 7. For any signed graph {G, Y), any K > 0, and any active learning algorithm A 
for link classification that queries the labels of a fraction a > of the edges of G, there exists 
a randomized labeling such that the number M of mistakes made by A in the prediction phase 
satisfies ¥.M > ^K, while A2(F) < K. 

Comparing this bound to that in Theorem |2]reveals that the active learning lower bound seems 
to drop significantly. Indeed, because the two learning protocols are incomparable (one is passive 
online, the other is active batch) so are the two bounds. Besides, Theorem [2] depends on A(F) 
while Theorem|7]depends on the larger quantity A2{Y). Next, we design and analyze two efficient 
active learning algorithms working under different assumptions on the way edges are labeled. 
Specifically, we consider two models for generating labelings Y: p-random and adversarial. In the 
p-random model, an auxiliary labeling Y' is arbitrarily chosen such that A2(F') = 0. Then Y is 
obtained through a probabilistic perturbation of Y', where f(Ye ^ FJ) < p for each e E E (note 
that correlations between flipped labels are allowed) and for some p E [0, 1). In the adversarial 

A very similar analysis can be carried out using A2 instead of A. In this case the uniform convergence term is of 
the form ^\E\ \V\. 
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model, Y is completely arbitrary, and corresponds to an arbitrary partition of V made up of two 
clusters. 

4.1 Random labeling 

Let Efiip denote the subset of edges e E E such that Fg 7^ '^he p-random model. The bounds 
we prove hold in expectation over the perturbation of Y' and depend on E|i5flip| rather than A2{Y). 
Clearly, since each label flip can increase A2 by at most one, then A2(F) < l-Eflip]. Moreover, one 
can show (details are omitted from this version of the paper) that there exist classes of dense graphs 
on which I-Empl = ^2 with high probability. 

During the selection phase, our algorithm for the p-random model queries only the edges of a 
spanning tree T = (Vr, Et) of G. In the prediction phase, the label of any remaining test edge 
e' = ^ Et is predicted with the sign of the product over all edges along the unique path 

PathT(e') between i and j in T. Clearly, if a test edge e' is predicted wrongly, then either e' E EfHp 
or Path7-(e') contains at least one edge of -Eflip. Hence, the number of mistakes Mt made by our 
active learner on the set of test edges E \ Et can be deterministically bounded by 

MT<|^flip|+ $^l{eePathT(e')}l{eeEflip} (3) 

where denotes the indicator of the Boolean predicate at argument. Let |Pathr(e') | denote the 
number of edges in Pathr(e'). A quantity which can be related to Mt is the average stretch of 

a spanning tree T which, for our purposes, reduces to (^\V\ — 1 + J2e'eE\ET\^^'^^Ti<^')\j ■ 
A beautiful result of [10] shows that every connected and unweighted graph has a spanning tree 
with an average stretch of just C(log^ \ V\ log log |V^|). Moreover, this low-stretch tree can be 
constructed in time In |\^|) . If our active learner uses a spanning tree with the same low 

stretch, then the following result can be easily proven. 

Theorem 8. Let {G, Y) be labeled according to the p-random model and assume the active learner 
queries the edges of a spanning tree T with average stretch 0(log^ \ V\ log log Then EM^ < 
p\E\ X C(log^ |1/|loglog|\/|) . 

4.2 Adversarial labeling 

The p-random model has two important limitations: first, depending on the graph topology, the 
expected size of Ef^p may be significantly larger than A2. Second, the tree-based active learning 
algorithm for this model works with a fixed query budget of | — 1 edges (those of a spanning tree). 
We now introduce a more sophisticated algorithm for the adversarial model which addresses both 
issues: it has a guaranteed mistake bound expressed in terms of A2 and works with an arbitrary 
budget of edges to query. 

Given (G, Y), fix an optimal two-clustering of the nodes with cost A2 = A2(F). Call S-edge 
any edge (i, j) G E whose sign Yij disagrees with this optimal two-clustering. Namely, Yij = — 1 
if i and j belong to the same cluster, or Yij = +1 if i and j belong to different clusters. Let E^^QE 
be the subset of 5-edges. 
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We need the following ancillary definitions and notation. Given a graph G = {Vg,Ec), and 
a rooted subtree T = (Vy, Et) of G, we denote by Tj the subtree of T rooted at node i E Vt- 
Moreover, if T is a tree and T' is a subtree of T, both being in turn subtrees of G, we let Eq{T\ T) 
be the set of all edges of Eq \ E^ that link nodes in Vt' to nodes in \ Vt'. Also, for nodes 
i,j E Vg, of a signed graph (G, F), and tree T, we denote by 7rr(?, j) the product over all edge 
signs along the (unique) path Pathr(i, j) between i and j in T. Finally, a circuit G = {Vc, Eq) 
(with node set C Vg and edge set Eq C i^c) of G is a cycle in G. We do not insist on the cycle 
being simple. Given any edge belonging to at least one circuit G = {Vc, Ec) of G, we let 
Gj j be the path obtained by removing edge (i, j) from circuit G. If G contains no (5-edges, then it 
must be the case that Yij = ttc^ . {i, j). 

Our algorithm finds a circuit covering C{G) of the input graph G, in such a way that each circuit 
G E C(G) contains at least one edge {ic,jc) belonging solely to circuit G. This edge is included 
in the test set, whose size is therefore equal to |C(G)|. The query set contains all remaining edges. 
During the prediction phase, each test label Yi^ j^ is simply predicted with TCd^ ^^{ic, jc)- See 
Figure [2] (left) for an example. 

For each edge let Lij be the the number of circuits of C(G) which (z, j) belongs to. We 
call Lij the load of {i, j) induced by C{G). Since we are facing an adversary, and each 5-edge may 
give rise to a number of prediction mistakes which is at most equal to its load, one would ideally 
like to construct a circuit covering C(G) minimizing max(j j)^^; Ljj, and such that |C(G)| is not 
smaller than the desired test set cardinality. 

Our algorithm takes in input a test set-to-query set ratio p and finds a circuit covering C(G) 
such that: (i) |C(G)| is the size of the test set, and (ii) gl'j[^|^^ > p, where Q is the size of the 

chosen query set, and (iii) the maximal load max(jj)g£;g Lj is 0{p^/'^ ^J\Vg\)- 

For the sake of presentation, we first describe a simpler version of our main algorithm. This 
simpler version, called SCCCC (Simplified Constrained Circuit Covering Classifier), finds a circuit 
covering C(G) such that max(j Ljj = 0{p^J\EG\), and will be used as a subroutine of the 
main algorithm. 

In a preliminary step, SCCCC draws an arbitrary spanning tree T of G and queries the labels of 
all edges of T. Then SCCCC partitions tree T into a small number of connected components of T. 
The labels of the edges (i, j) with i and j in the same component are simply predicted by 71^(2, j). 
This can be seen to be equivalent to create, for each such edge, a circuit made up of edge (i, j) and 
PathT(«, j). For each component T', the edges in Eg{T', T) are partitioned into query set and test 
set satisfying the given test set-to-query set ratio p, so as to increase the load of each queried edge 
in Et \ E^' by only 0{p). Specifically, each test edge (i, j) E Eq{T' , T) lies on a circuit made up 
of edge (i, j) along with a path contained in T', a path contained in T \ T', and another edge from 
Eg(T', T). a key aspect to this algorithm is the way of partitioning tree T so as to guarantee that 
the load of each queried edge is 0{p^J\EG\). 

In turn, SCCCC relies on two subroutines, TreePartition and EdgePartition, which we 
now describe. Let T be the spanning tree of G drawn in SCCCC's preliminary step, and T' be 
any subtree of T. Let v be an arbitrary vertex belonging to both Vt and Vt' and view both trees 
as rooted at TreePartition (T', G, 9) returns a subtree Tj of T such that: (i) for each node 
v^]of Tj we have |Eg(T^, T') | < 6, and (ii) |^G(Tj, T) | > ^. In the special case when no such 



subtree Tj exists, the whole tree T' is returned, i.e., we set Tj = T' . As we show in Lemma ^0 



in Appendix pi in this special case (i) still holds. TreePartition can be described as follows 
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Figure 2: Left: An illustration of how a circuit can be used in the selection and prediction phases. 
An optimal two-cluster partition is shown. The negative edges are shown using dashed (either 
thick black or thin gray) lines. Two circuits are depicted using thick black lines: one containing 
edge (ii, ji), the other containing edge {12, j2)- For each circuit C in the graph, we can choose 
any edge belonging to C to be part of the test set, all remaining edge labels being queried. In this 
example, if we select as test set edge, then label l^ui is predicted with (—1)^ = —1, since 

5 is the number of negative edges in Ec \ {(ii, ji)}. Observe that the presence of a 5-edge (the 
negative edge incident to ii) causes a prediction mistake in this case. The edge Yi^j^ on the other 
circuit is predicted correctly, since this circuit contains no 5-edges. Right: A graph G = {V, E) 
and a spanning tree T = (Vt, Ex), rooted at ir, whose edges Et are indicated by thick lines. A 
subtree Tj, rooted at j, with grey nodes. According to the level order induced by root ir, nodes 
j[, j'2 and are the children of node j. TreePartition computes EciTj, T), the set of edges 
connecting one grey node to one white node, as explained in the main text. Since \EG{Tj, T)\ > 9, 
TreePartition invoked with = 9 returns tree Tj. 

— see Figure [2] (right) for an example. We perform a depth-first visit of the input tree starting from 
V. We associate some of the nodes i G Vt' with a record Ri containing all edges of Eg{TI, T'). 
Each time we visit a leaf node j, we insert in Rj all edges linking j to all other nodes in T' (except 
for j's parent). On the other hand, if j is an internal node, when we visit it for the last timej^ we 
set Rj to the union of Rji over all j's children j' and jg in Figure |2] (right)), excluding the 

edges connecting the subtrees T,/ to each other. For instance, in Figure pi (right), we include all 
gray edges departing from subtree Tj, but exclude all those joining the three dashed areas to each 
other. In both cases, once Rj is created, if > or j = ir, TreePartition stops and returns 
Tj. Observe that the order of the depth-first visit ensures that, for any internal node j, when we are 
about to compute Rj, all records Rj' associated with j's children j' are already available. 

We now move on to describe EdgePartition. Let ir q e Vt'. EdgeParti- 
TlON(Tg, T', G, p) returns a partition £ = {Ei,E2, . . .} of EciT'^, T') into edge subsets of car- 
dinality p + 1, that we call sheaves. If \Eg{T'^,T') \ is not a multiple of p + 1, the last sheaf can 
be as large as 2p. After this subroutine is invoked, for each sheaf E^ G 8, SCCCC queries the 
label of an arbitrary edge (z, j) G E^, where i G Vt^ and j G Vj" \ Vr^. Each further edge 
(z', j') G -Efc \ {(z, j)}, having i' G and j' G Vt', will be part of the test set, and its label 

^ Since j is an internal node, its last visit is performed during a backtracking step of the depth-first visit. 



10 



SCCCC(p, 9) Parameters: p> 0,9 > 1. 

1 . Draw an arbitrary spanning tree T of G, and query all its edge labels 

2. Do 

3 . ^ TreePartition(T, G, 9) 

4 . For each i,j e Vt^, set Yij ^ -KTii,]) 

5 . £ ^ EDGEPARTITION(rg, T, G, p) 

6 . For each Ek e £ 

I . query the label of an arbitrary edge (i, j) G -E^ 

8 . For each edge {i\ f) eEk\{{i,3)}, where i, i' e Vt^ and j, f eVtX Vt^ 

9 . ^ 71t{i', i) ■ Yij ■ Mj, J ) 
10. T^T\Tq 

II. While(\/T ^0) 



Figure 3: The Simplified Constrained Circuit Covering Classifier SCCCC. 



will be predicted using the pathj^Path7-(i', i) — )■ — )■ Path2-(j,j') which, together with test 
edge forms a circuit. Note that all edges along this path are queried edges and, moreover, 

Lij < 2p) because cannot belong to more than 2p circuits of C{G). Label Fj'j/ is therefore 
predicted by TTrii', i) ■ Yij ■ tttU,]'). 

From the above, we see that Eg{T'^,T') is partioned into test set and query set with a ratio 
at least p. The partition of E(T^,T') into sheaves is carefully performed by EdgePartition 
so as to ensure that the load increse of the edges in E^' \ -Et^ is only 0{p), independent of the 
size of E{T'^, T'). Moreover, as we show below, the circuits of C{G) that we create by invoking 
EdgePartition (T^, T', G, p) increase the load of each edge of (m, v) (where u is parent of v in 
Tg) by at most 2p \Eg{T!^-,T')\. This immediately implies — see Lemma 101) in Appendix |b] — 



that if Tg was previously obtained by calling TREEPARTlTlON(r', G, 9), then the load of each edge 
of T'q gets increased by at most 2p9. We now describe how EdgePartition builds the partition of 
Eg(T^, T') into sheaves. EDGEPARTlTlON(Tg, T', G, p) first performs a depth-first visit of V \ 
starting from root Then the edges of Eg{T^, T') are numbered consecutively by the order of 
this visit, where the relative ordering of the edges incident to the same node encountered during 
this visit can be set arbitrarily. Figure |5] in Appendix |B] helps visualizing the process of sheaf 
construction. One edge per sheaf is queried^the remaining ones are assigned to the test set. 

SCCCC's pseudocode is given in Figure pi Y^j therein denoting the predicted labels of the test 
set edges («, j). The algorithm takes in input the ratio parameter p (ruling the test set-to-query 
set ratio), and the threshold parameter 9. After drawing an initial spanning tree T of G, and 
querying all its edge labels, SCCCC proceeds in steps as follows. At each step, the algorithm calls 
TreePartition on (the current) T. Then the labels of all edges linking pairs of nodes i, j G Vt^ 
are selected to be part of the test set, and are simply predicted by Yij ^ 7rT{i,j). Then, all edges 
linking the nodes in Vt^ to the nodes in Vj- \ Vig are split into sheaves via EdgePartition. For 
each sheaf, an arbitrary edge (i, j) is selected to be part of the query set. All remaining edges (i', j') 
become part of the test set, and their labels are predicted by Yi'j/ ^ 7rr(«', i) ■ Yij ■ titU, j'), where 
i, i' E Vt and j,j' E Vt\ Vt . Finally, we shrink T as T \ Tg, and iterate until Vr = 0- Observe 



' Observe that PathT^(j, j') = PathT'(j,/) = PathrijJ'), since C T' C T. 
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CCCC(p) Parameter: p satisfying 3 < p < 

1 . Initialize E ^ Eg 

2. Do 

3 . Select an arbitrary edge subset E' C E such that = min{|_E|, p| Vg|} 
4. LetG' = (^G,^') 

5 . For each connected component G" of G', run scccc(p, a/I-E'I) on G" 
6. E i- E\E' 
1 . While (E ^ 0) 

Figure 4: The Constrained Circuit Covering Classifier CCCC. 



that in the last do-while loop execution we have T = TREEPARTlTlON(r, G, 0). Moreover, if 
Tg = T, Lines 6-9 are not executed since E^Tq, T) = 0, which implies that £ is an empty set 
partition. 

We are now in a position to describe a more refined algorithm, called CCCC (Constrained 
Circuit Covering Classifier — see Figure |4]), that uses SCCCC on suitably chosen subgraphs of 
the original graph. The advantage of CCCC over SCCCC is that we are afforded to reduce the 



mistake bound from 0{A2{Y) p ^/\Eg\) (Lemma [13| in Appendix |b|) to 0{A2(Y) p2 vlW)- 
CCCC proceeds in (at most) \Eg\/ (p| Vg|) steps as follows. At each step the algorithm splits into 
query set and test set an edge subset E' C E, where E is initially Eg- The size of E' is guaranteed 
to be at most p| Vbl- The edge subset E' is made up of arbitrarily chosen edges that have not been 
split yet into query and test set. The algorithm considers subgraph G' = {Vg,E'), and invokes 
SCCCC on it for querying and predicting its edges. Since G' can be disconnected, CCCC simply 
invokes SCCCC(p, ^/\E'\) on each connected component of G'. The labels of the test edges in E' 
are then predicted, and E is shrunk to E\ E'. The algorithm terminates when E = 

Theorem 9. The number of mistakes made by CCCC(p), with p satisfying 3 < p < on a graph 

G = (Vg, Eg) with unknown labeling Y is C(A2(F)pt a/|Vg|). Moreover, we have ^-^^ > 
where Q is the size of the query set and \C{G) \ is the size of the test set. 

Remark 1. Since we are facing a worst-case (but oblivious) adversary, one may wonder whether 
randomization might be beneficial in SCCCC or CCCC. We answer in the affermative as follows. 
The randomized version o/SCCCC is SCCCC where the following two steps are randomized: (i) The 
initial spanning tree T ( Line 1 in Figure is drawn at random according to a given distribution 
T> over the spanning trees of G. (ii) The queried edge selected from each sheaf E^ returned by 
calling EdgePartition (Line 7 in Figure^ is chosen uniformly at random among all edges in 
Ek. Because the adversarial labeling is oblivious to the query set selection, the mistake bound of 
this randomized SCCCC can be shown to be the sum of the expected loads of each 6-edge, which 
can be bounded by o(^A2{Y) max{l, pP^'''' a/I-^gI — \Vg\ + where P.^"^^ is the maximal 
over all probabilities of including edges G E in T. When T is a uniformly generated random 
spanning tree 123], and p is a constant (i.e., the test set is a constant fraction of the query set) 
this implies optimality up to a factor kp (compare to Theorem^ on any graph where the effective 
resistance / |23]/ between any pair of adjacent nodes in G is 0[k/\V\) —for instance, a very dense 
clique-like graph. One could also extend this result to CCCC, but this makes it harder to select the 
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parameters of SCCCC within CCCC. 



We conclude with some remarks on the time/space requirements for the two algorithms SC- 
CCC and CCCC, details will be given in the full version of this paper. The amortized time 

per prediction required by scccc(p, y/\Ec\ - \Vg\ + 1) and cccc(p) is o(-^^L log |Vb| ) and 

C^-y/i^'log |Vf7| respectively, provided = ^1{\Eg\) and p < a/| Vg|. For instance, when 

the input graph G = (Vg, Eg) has a quadratic number of edges, SCCCC has an amortized time 
per prediction which is only logarithmic in \ Vg\. In all cases, both algorithms need linear space in 
the size of the input graph. In addition, each do-while loop execution within CCCC can be run in 
parallel. 



5 Conclusions and ongoing research 

In this paper we initiated a rigorous study of link classification in signed graphs. Motivated by 
social balance theory, we adopted the correlation clustering index as a natural regularity measure 
for the problem. We proved upper and lower bounds on the number of prediction mistakes in 
three fundamental transductive learning models: online, batch and active. Our main algorithmic 
contribution is for the active model, where we introduced a new family of algorithms based on 
the notion of circuit covering. Our algorithms are efficient, relatively easy to implement, and have 
mistake bounds that hold on any signed graph. We are currently working on extensions of our 
techniques based on recursive decompositions of the input graph. Experiments on social network 
datasets are also in progress. 
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A Proofs 

LemmaUl The edge set of a clique G = (V, E) can be decomposed into edge-disjoint triangles if 
and only if there exists an integer /c > Osuchthat|K| = 6A;+lor|K| = 6k+3 — see, e.g., page 1 13 
of Bill. This implies that for any clique G we can find a subgraph G' = {V, E') such that G' is a 
clique, \ V'\ > \V\ — 3, and E' can be decomposed into edge-disjoint triangles. As a consequence, 
we can find K edge-disjoint triangles among the = || V|(| V| - 1) > - 3)(|V| - 4) 

edge-disjoint triangles of G', and label one edge (chosen arbitrarily) of each triangle with —1, all 
the remaining \E\ — K edges of G being labeled +1. Since the elimination of the K edges labeled 
—1 implies the elimination of all bad cycles, we have A(F) < K. Finally, since A(F) is also 
lower bounded by the number of edge-disjoint bad cycles, we also have A(F) > K. □ 

Theorem^ The adversary first queries the edges of a spanning tree of G forcing a mistake at each 
step. Then there exists a labeling of the remaining edges such that the overall labeling Y satisfies 
A(F) = 0. This is done as follows. We partition the set of nodes V into clusters such that each 
pair of nodes in the same cluster is connected by a path of positive edges on the tree. Then we label 
+1 all non-tree edges that are incident to nodes in the same cluster, and label —1 all non-tree edges 
that are incident to nodes in different clusters. Note that in both cases no bad cycles are created, 
thus A(F) = 0. After this first phase, the adversary can force additional K mistakes by querying 
K arbitrary non-tree edges and forcing a mistake at each step. Let Y' be the final labeling. Since 
we started from Y such that A(F) = and at most K edges have been flipped, it must be the case 
that A(F') <K. □ 

Theorem \3\ If > ^ we have 1 | + -ft' > | V | + -ft' (log2 ^ — 2 j , so one can prove the statement 
just by resorting to the adversarial strategy in the proof of Theorem [2} Hence, we continue by 
assuming -ft' < We first show that on a special kind of graph G' = {V, E'), whose labels 
Y' are partially revealed, any algorithm can be forced to make at least logg \V'\ mistakes with 
A(y) = 1. Then we show (Phase 1) how to force \V\ — 1 mistakes on G while maintaining 
A(F) = 0, and (Phase 2) how to extract from the input graph G, consistently with the labels 
revealed in Phase 1, K edge-disjoint copies of G' . The creation of each of these subgraphs, which 



contain \V'\ = 2Li°g2(|v|/(2X))j nodes, contributes 



log; 



2 2K 



> log2 ^ — 2 additional mistakes. In 



Phase 2 the value of A(F) is increased by one for each copy of G' extracted from G. 
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Let d{i,i) be the distance between node i and node j in the graph under consideration, i.e., 
the number of edges in the shortest path connecting i to j. The graph G' = {V',E') is con- 
structed as follows. The number of nodes \V'\ is a power of 2. G' contains a cycle graph G 
having \ V'\ edges, together with \V'\/2 additional edges. Each of these additional edges connects 
the \ V'\/2 pairs of nodes {io,jo}, ■ ■ ■ ofV such that, for all indices A; > 0, the distance 

d{ik,jk) calculated on G is equal to \V'\/2. We say that ik and jk are opposite to one another. 
One edge of G, say (zo,«i), is labeled —1, all the remaining \V'\ — 1 in C are labeled +1. All 
other edges of G', which connect opposite nodes, are unlabeled. We number the nodes of V as 
io, zi, . . . , V'l/2-i'io, ji, • • • , j|y'|/2-i in such a way that, on the cycle graph G, 4 and are adja- 
cent to ik^i and jk~i, respectively, for all indices k >0. With the labels assigned so far we clearly 
have A (F') = 1. 

We now show how the adversary can force log2 \ V'\ mistakes upon revealing the unassigned 
labels, without increasing the value of A(y). The basic idea is to have a version space S' of G', 
and halve it at each mistake of the algorithm. Since each edge of G can be the (unique) 5-edgej^we 
initially have |S"| = \V'\. The adversary forces the first mistake on edge {io,jo), just by assigning 
a label which is different from the one predicted by the algorithm. If the assigned label is +1 then 
the 5-edge is constrained to be along the path of G connecting io to jo via ii, otherwise it must 
be along the other path of G connecting iQ to jo- Let now L be the line graph including all edges 
that can be the 5-edge at this stage, and u be the node in the "middle" of L, (i.e., u is equidistant 
from the two terminal nodes). The adversary forces a second mistake by asking for the label of the 
edge connecting u to its opposite node. If the assigned label is +1 then the S-edge is constrained 
to be on the half of L which is closest to io, otherwise it must be on the other half. Proceeding 
this way, the adversary forces log2 \ V'\ mistakes without increasing the value of A(y). Finally, 
the adversary can assign all the remaining \ V'\/2 — logg \ V'\ labels in such a way that the value of 
A(y) does not increase. Indeed, after the last forced mistake we have \S'\ = 1, and the 5-edge is 
completely determined. All nodes of the labeled graph obtained by flipping the label of the 5-edge 
can be partitioned into clusters such that each pair of nodes in the same cluster is connected by a 
path of +l-labeled edges. Hence the adversary can label all edges in the same cluster with +1 and 
all edges connecting nodes in different clusters with —1. Clearly, dropping the 5-edge resulting 
from the dichotomic procedure also removes all bad cycles from G'. 

Phase 1. Let now H be any Hamiltonian path in G. In this phase, the labels of the edges in H 
are presented to the learner, and one mistake per edge is forced, i.e., a total of |y| — 1 mistakes. 
According to the assigned labels, the nodes in V can be partitioned into two clusters such that any 
pair of nodes in each cluster are connected by a path in H containing an even number of — 1-labeled 
edges Let now Vq be the larger cluster and vi be one of the two terminal nodes of H. We number 
the nodes of Vq as t>i, t>2, . . . , in such a way that Vk is the k-th node closest to vi on H. Clearly, all 
edges (f fc, Vk+i) either have been labeled +1 in this phase or are unlabeled. For all indices k > 0, 
the adversary assigns each unlabeled edge {vk, Vk+i) of G a +1 label. Note that, at this stage, no 
bad cycles are created, since the edges just labeled are connected through a path containing two 
— 1 edges. 

Phase 2. Let Hq be the line graph containing all nodes of Vq and all edges incident to these 
nodes that have been labeled so far. Observe that all edges in Hq are +1. Since |Vo| > |V^|/2, 

^ Here a 6-edge is a labeled edge contributing to A{Y). 

^ In the special case when there is only one cluster, we can think of the second cluster as the empty set. 
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Hq must contain a set of K edge-disjoint line graphs having 2L^°S2(I^I/(^^))J edges. The adversary 
then assigns label —1 to all edges of G connecting the two terminal nodes of these sub-line graphs. 
Consider now all the cycle graphs formed by all node sets of the K sub-line graphs created in the 
last step, together with the —1 edges linking the two terminal nodes of each sub-line graph. Each 
of these cycle graphs has a number of nodes which is a power of 2. Moreover, only one edge is —1, 
aU remaining ones being +1. Since no edge connecting the nodes of the cycles has been assigned 
yet, the adversary can use the same dichotomic technique as above to force, for each cycle graph. 



log; 



M 

2 2K 



additional mistakes without increasing the value of A(F). □ 



Theorem^ We first claim that the following bound on the version space size holds: 

log2|5*| <rf*l0g2^ + |l^|l0g2 



d* ' ' ' \n{\V\ + 1) ■ 

To prove this claim, observe that each element of S* is uniquely identified by a partition of V 
and a choice of d* edges in E. Let i?„ (the Bell number) be the number of partitions of a set of 

n elements. Then 15*1 < B\y\ x ('^') . Using the upper bound < j and standard 

binomial inequalities yields the claimed bound on the version space size. 

Given the above, the mistake bound of the resulting algorithm is an easy consequence of the 
known mistake bounds for Weighted Majority. □ 

Theorem^ sketch. We start by showing that the Halving algorithm is able to solve UCC by build- 
ing a reduction from UCC to link classification. Given an instance of {G, Y) of UCC, let the 
supergraph G' = {V, E') be defined as follows: introduce G" = (V", E"), a copy of G, and let 
V = VUV". Then connect each node i" G V" to the corresponding node i E V, add the resulting 
edge (i, i") to E' , and label it with +1. Then add to E' all edges in E, retaining their labels. Since 
the optimal clustering of (G, Y) is unique, there exists only one assignment Y" to the labels of 
E" C E' such that A(y) = A(F). This is the labeling consistent with the optimal clustering C* 
of (G, Y): each edge [i" , j") E E" is labeled +1 if the corresponding edge (z, j) E E connects two 
nodes contained in the same cluster of C* , and —1 otherwise. Clearly, if we can classify correctly 
the edges of E" , then the optimal clustering C* is recovered. In order to do so, we run HAL^i on 
G" with increasing values of d starting from c? = 0. For each value of d, we feed all edges of 
G" to HALd and check whether the number z of edges (i", j") E E" for which the predicted label 
is different from the one of the corresponding edge (i, j) E E, is equal to d. If it does not, we 
increase d by one and repeat. The smallest value d* of d for which z is equal to d must be the true 
value of A(y). Indeed, for all d < d* Halving cannot find a labeling of G' with cost d. Then, we 
run HALrf. on G' and feed each edge of E". After each prediction we reset the algorithm. Since the 
assignment Y" is unique, there is only one labeling (the correct one) in the version space. Hence 
the predictions of H AL^. are all correct, revealing the optimal clustering C* . The proof is concluded 
by constructing the series of reductions 

Unique Maximum Clique — )• Vertex Cover — t- Multicut — )• Correlation Clustering, 

where the initial problem in this chain is known not to be solvable in polynomial time, unless 
RP = NP. □ 
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Theorem^ First, by a straightforward combination of [9, Remark 2] and the union bound, we 
have the following uniform convergence result for the class V: With probability at least 1 — 5, 
uniformly over / G P, it holds that 



-A„(/) < -A„(/) + cj(- + -] (\V\ \n\V\+ In (4) 

where c is a suitable constant. Then, we let /* G P be the partition that achieves Am+„(/*) = A. 
By applying Remark 3] we obtain that 



-A„(/) < -A„(r) < ^A„+„(r) + c'«:J^ln^ 
m m m + u \ m + u o 



with probability at least 1 — f • An application of (|4j) concludes the proof. □ 

Theorem^ Let Y be the following randomized labeling: All edges are labeled +1, except for a 
pool of K edges, selected uniformly at random, and whose labels are set randomly. Since the size 
of the training set chosen by A is not larger than a\E\, the test set will contain in expectation at 
least (1 — a)K randomly labeled edges. Algorithm A makes in expectation 1/2 mistakes on every 
such edge. Now, if we delete the edges with random labels we obtain a graph with all positive 
labels, which immediately implies A2(F) < K. □ 

Theorem^ We start from ([3]) and take expectations. We have 

EMt < p\E\ + "^Ilee PathT(e')}P(e G ^mp) 

= p\E\+p ^ |PathT(e')| 

e'eE\ET 

= p\E\ +p\E\ X C(log^ |1/|loglog|y|) 
= p\E\ X C(log^ \V\ log log |V"|) 

as claimed. □ 



B Additional figures, lemmas and proofs from Section |4| 

Lemma 10. Let T' be any subtree ofT, rooted at (an arbitrary) vertex v £ Vt'- If for any node 
V G Vt' we have \Ec{T^,T')\ < 9, then TreePartition(T', G, 6') returns T itself otherwise, 
TREEPARTlTlON(r', G, 9) returns a proper subtree T- C T' satisfying 

(i) lEciT!^, T')\ < 9 for each node v ^ j in Vt'., and 

(ii) \Eg{T'T')\>0. 
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Figure 5: An illustration of how EdgePartition operates and how the SCCCC prediction rule 
uses the output of EdgePartition. Notation is as in the main text. In this example, EdgePar- 
tition is invoked with parameters T'^,T' , G, 3. The nodes in are grey. The edges of Et' are 
thick black. A depth-first visit starting from ir is performed, and each node is numbered according 
to the usual visit order. Let ii, ^2, • • • be the nodes in V^' \ (the white nodes), and ji, j2, • • • 
be the nodes of Vt^ (the gray nodes). The depth-first visit on the nodes induces an ordering on 
the 8 edges of Eg{T'^-,T') (i.e., the edges connecting grey nodes to white nodes), where 
precedes (i2, 32) if h is visited before 12, being ii, ^2, ■ • ■ and ji, j2, ■ ■ ■ belonging to V^'\t^ and 
respectively. The numbers tagging the edges of Eg{T'^,T') denote a possible edge ordering. In 
the special case when two or more edges are incident to the same white node, the relative order 
for this edge subset is arbitrary. This way of ordering edges is then used by EdgePartition for 
building the partition. Since p = 3, EdgePartition partitions EciT'^^T') in 8/(3 + 1) = 2 
sheaves, the first one containing edges tagged 1, 2, 3, 4 and the second one with edges 5, 6, 7, 8. 
Finally, from each sheaf an arbitrary edge is queried. A possible selection, shown by grey shaded 
edges, is edge 1 for the first sheaf and edge 5 for the second one. The remaining edges are as- 
signed to the test set. For example, during the prediction phase, test edge (19, 26) is predicted as 
^19,26 ^ >^i9,7>7,24>24,25^25,26, whcrc Yi9,7, 124,25, and 1^25,26 are available since they belong to the 
initial spanning tree T' . 

Proof. The proof immediately follows from the definition of TreePartition(T", G, ^). If 
\Eg{T'^,T')\ < 6 holds for each node v e Vt', then TreePartition(T', G, 6*) stops only after 
all nodes of Vt' have been visited, therefore returning the whole input tree T'. On the other hand, 
when \Eg{T^, T')\ <9 does not hold for each node v eVr', TreePartition(T', G, 9) returns a 
proper subtree Tj c T'and,bythevery waythis subroutine works, we musthave |£^(5(Tj,T')| > 9. 
This proves (ii). In order to prove (i), assume, for the sake of contradiction, that there exists a node 
f ^ j of Vt' such that \Eg{T'^, T')\ > 9. Since f is a descendent of j, the last time when v gets 
visited precedes the last time when j does, thereby implying that TreePartition would stop at 
some node z of Vr^, which would make TreePartition(T', G, 9) return instead of Tj. □ 

Let C(T^,T', G,p) C C{G) be the set of circuits used during the prediction phase that have 
been obtained thorugh the sheaves {Ei, E2, . . .} returned by EdgePartition(T^, T', G, p). The 
following lemma quantifies the resulting load increase of the edges in Et' \ Et' . 
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Lemma 11. Let T' be any subtree ofT, rooted at (an arbitrary) vertex ir G Vt'- Then the load 
increase of each edge in E^' \ Et' resulting from using at prediction time the circuits contained in 
Cm,T',G,p)isO{p). 

Proof. Fix a sheaf E' and any circuit C G C(Tq, T', G, p) containing the unique queried edge of 
E' , and C be the part of C that belongs to T' \T'^. We know that the edges of C are potentially 
loaded by all circuits needed to cover the sheaf, which are 0{p). We now check that no more 
than 0{p) additional circuits use those edges. Consider the line graph L created by the depth first 
visit of T starting from ir. Each time an edge (z, j) is traversed (even in a backtracking step), the 
edge is appended to L, and j becomes the new terminal node of L. Hence, each backtracking step 
generates in L at most one duplicate of each edge in T, while the nodes in T may be duplicated 
several times in L. Let and Vax be the nodes of \ Vt' incident to the first and the last 
edge, respectively, assigned to sheaf E' during the visit of T, where the order is meant to be 
chronological. Let £min and fmax be the first occurrence of imin and imax in L, respectively, when 
traversing L from the first node inserted. Let L' be the sub-line of L having ^min and £max as 
terminal nodes. By the way EdgePartition is defined, all edges of C that are loaded by circuits 
covering E' must also occur in L' . Since each edge of T occurs at most twice in L, each edge of C 
belongs to L' and to at most another sub-line L" of L associated with a different sheaf E" . Hence 
the overall load of each edge in C is 0{p). □ 

We are now ready to bound the number of mistakes made when SCCCC is run on any labeled 
graph 

Lemma 12. The load of each queried edge selected by running SCCCC(p, 6) on any labeled graph 

(G = (Vfe, Eg). Y) is o {^p{\-^otm±i + e)). 

Proof. In the proof we often refer to the line numbers of the pseudocode in Figure [3] We start 
by observing that the subtrees returned by the calls to TreePartition are disjoint (each Tq is 
removed from T in line 10). This entails that EdgePartition is called on disjoint subsets of 
Eg, which in turn implies that the sheaves returned by calls to EDGEPARTITION are also disjoint. 
Hence, for each training edge (i, j) E Eg \ Ex (i.e., selected in line 7), we have Lj j = 0{p). This 
is because the number of circuits in C{G) that include is equal to the cardinality of the sheaf 
to which belongs (lines 8 and 9). 

We now analyze the load L^, of each edge (f , w) E Ex. This quantity can be viewed as the 
sum of three distinct load contributions: L^^^ = + ^ + L"'^. The first term accounts 
for the load created in line 4, when v and w belong to the same subtree Tg returned by calling 
TreePartition in line 3. The other two terms L"^ and L"'^ take into account the load created 
in line 9, when either v and w both belong (L" ^) or do not belong (L'^'^) to the subtree Tg returned 
in line 3. 

Assume now that both v and w belong to Vr^ with Tg returned in line 3. Without loss of 
generality, let v be the parent of w in T. The load contribution L[ ^ deriving from the circuits 
in C{G) that are meant to cover the test edges joining pairs of nodes of Tg (line 4) must then be 



bounded by |£'g(T^„, Tg) | . This quantity can in turn be bounded by 6 using part (i) of Lemma 10 
Hence, we must have L'^ ^ < 6'. 
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Observe now that L"^ may increase by one each time line 9 is executed. This is at most 



0{p) X \EG{Ty„Tg)\. Since \EG{T^,Tq)\ < 6 hy part (i) of Lemma 10 we must have L'^^ = 
0{pe). 

We finally bound the load contribution As we said, this refers to the load created in 

line 9 when neither v nor w belong to subtree Tq returned in line 3. Lemma 1 1 ensures that, 
for each call of EdgePartition, L"'^ gets increased by 0{p). We then bound the number of 
times when EdgePartition may be called. Observe that \EG{Tq,T)\ > 9 for each subtree 
Tq returned by TreePartition. Hence, because different calls to EdgePartition operate on 
disjoint edge subsets, the number of calls to EdgePartition must be bounded by the number 
of calls to TreePartition. The latter cannot be larger than \^g\-\Vg\+i ^j^^^h, in turn, implies 

L':,^ = o[li\EG\-\VG\ + l)). 
Combining together, we find that 

L„,^ = l;^ + 4',^ + l:;^ = o(e + p9 + ^{{EgI - \Vg\ + 1)) 

thereby concluding the proof. □ 



The value of threshold 9 that minimizes the above upper bound is ^ = a/|-Eg| — I VgI + 1, as 
exploited next. 

Lemma 13. The number of mistakes made by SCCCC(p, a/I-E'gI ^ I^gI + 1) on a labeled graph 
[G = {Vg, Eg), Y) is o(A2(r) p ^I^gI - \Vg\ + l), while we have g^^jVl ^ P' ^^^'"^ Q 
the size of the chosen query set (excluding the initial \ Vg\ — 1 labels), and \C{G) \ is the size of the 
test set. 

Proof The condition qI'^^^i+i > P immediately follows from: 

(i) the very definition of EdgePartition, which selects one queried edge per sheaf, the cardi- 
nality of each sheaf being not smaller than p + 1, and 

(ii) the very definition of SCCCC, which queries the labels of |Vg| — 1 edges when drawing the 
initial spanning tree T of C 

As for the mistake bound, recall that for each queried edge (i, j), the load Lj j of (i, j) is defined 
to be the number of circuits of C(G) that include {i,j). As already pointed out, each 5-edge cannot 
yield more mistakes than its load. Hence the claim simply follows from Lemma 12 and the chosen 
value of □ 

Theorem^ In order to prove the condition ^^^q^^ > it suffices to consider that: 

(i) a spanning forest containing at most |Vg| — 1 queried edges is drawn at each execution of 
the do-while loop in Figure |4} and 

(ii) EdgePartition queries one edge per sheaf, where the size of each sheaf is not smaller 
than p + 1. Because j^^^ + 1 bounds the number of do-while loop executions, we have that 



21 



the number of queried edges is bounded by 



< !^ + \Va\ + ^""^ 
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where in the last inequality we used p < Hence ^^^^^ = ^'^'^g"'^ > as claimed. 

Let us now turn to the mistake bound. Let Vg" and Eg" denote the node and edge sets of the 
connected components G" of each subgraph G' C G on which CCCC(p) invokes SCCCC(p, \fE'\ 
By Lemma 12 we know that the load of each queried edge selected by SCCCC on G" is bounded 
by 



G 



the first equality deriving from |-Eg"| "~ I^C'I + 1 < l-^^'l^ and the second one from \E'\ < pIVgI 
(line 3 of CCCC's pseudocode). 

Li order to conclude the proof, we again use the fact that any 5-edge cannot originate more 
mistakes than its load, along with the observation that the edge sets of the connected component 
G" on which SCCCC is run are pairwise disjoint. □ 
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